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1. Introduction 


Recently, empirical reports from the Brazilian National Council of Justice 
(CNJ) have shed light on the numbers of the Brazilian Courts. There is a 
backlog of almost 80 million lawsuits waiting for a final and binding decision 
(1 for every 2.6 inhabitants). Additionally, Brazil spends almost 2% of its 
Gross Domestic Product (GDP) with the Judiciary Branch, which is more 


than any other country in the world.? 


There were many attempts to tackle the crisis of the justice system; 
however, all of them have failed. For instance, despite the efforts to 
promote alternative dispute resolution (ADR) mechanisms in Brazil, to date 


only 10% of the lawsuits end in a settlement. 


The implementation of technology in the Brazilian Courts seems to be the 
only way out. Adopting cutting-edge technology to solve legal conflicts, 
therefore, is the major agenda of public and private stakeholders on the 
field?, and Brazil has experienced a huge growth in the number of legal 
technology companies as well as in the digitalization of courts and 


tribunals.3 


In this paper, we decided to highlight the initiatives carried out by the 
Judiciary Branch to establish partnerships with researchers for the use of 


artificial intelligence (AI) that assists in the classification and identification 
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of claims, and even in the performance of the administrative acts that are 


necessary for the trial of repetitive cases. 


Notably, the Brazilian Supreme Court (STF) recently announced the creation 
of an AI system, designed in a partnership with the University of Brasilia 
(UnB), named VICTOR. The AI’s nickname is a tribute to the deceased 
Minister Victor Nunes Leal, the former Justice responsible for the 


systematization of STF’s case law. 


Referred to as the 12" Justice of the STF, the AI system, in its initial phase, 
has the goal of analysing the huge load of appeals presented before the 
High Court and to automate the evaluation of one of its prerequisites: 
Victor’s pilot version aims to identify cases with repercussão geral (“general 
repercussion”), a requirement for the processing of an appeal before the 
STF, which will be further explained.* 


This short paper aims to present VICTOR and discuss its advantages, flaws, 
and risks, as well as answer the playful question posed in the title: is 
VICTOR a beauty or a beast? 


2. How does VICTOR work? 


Before we address how VICTOR’s operates, it is necessary to make some 
introductory considerations regarding AI. We adopt Pedro Domingos' 
definition, valuable for its simplicity: an algorithm is a sequence of 


instructions that tells a computer what to do.” 


Regarding its operation, we can divide the algorithms into two species: 
deterministic and probabilistic. Deterministic algorithms follow operations 
(the path) defined by the programmer. Thus, the information is carried in 
the system (input), the algorithm does what is programmed to do with it, 


and the result (output) “comes out” from the system. 
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Referring to the operation of deterministic algorithms, Alan Turing, in the 
seminal Computing Machinery and Intelligence, written in 1950, proposed 
that, instead of simulating the brain of a human adult and programming all 
the operations to be performed, it would be more productive to adopt a 
different strategy: simulating a child's brain with random learning. This is 


how probabilistic algorithms work. 


These algorithms operate creating other algorithms: the data and the 
desired result are inserted into the system (input), and it produces the 
algorithm (output) that allows the transformation of the former into the 
latter. As Pedro Domingos points out, the computer creates its own code so 


that humans do not have to do it.® 


Machine learning techniques can then be defined as the practice of using 
algorithms to collect, interpret, and make predictions based on data. Since 
they “feed” the system, being the raw material from which the software 
bases its learning activity, a huge amount of data is essential for the 


machine learning to achieve its full potential. 7 


The simplest form of machine learning is one that employs supervised 
algorithms, in which the system is fed with selected and structured data. In 
this case, the dataset and the expected output are loaded into the system. 
While being trained, the machine learning model adjusts its variables to 


relate the inputs to the corresponding output. 


The credit score carried out by financial institutions is a good example: the 
data analysed will refer to the customer's credit history, and the information 
used to train the system is data that had already been classified whether 


as positive or negative regarding the credit evaluation.’ 
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A sort of algorithmic structure that works in a supervised way is the artificial 
neural network with backpropagation.!° Inspired by the human brain, these 
artificial networks have learning models based on mistakes and successes, 
with gradual identification of the best paths and decisions to achieve certain 


objectives. 


The system is loaded with a single objective (output) and several inputs, 
which are tested in several ways. When the desired outcome is achieved, 
the most assertive path receives a greater weight (in the mathematical 
account). Thus, the internal neural layers (hidden layers) start to deliver 


more accurate results.?! 


A second relevant category is the non-supervised algorithms. In this case, 
the data that feeds the system is not classified, leaving the algorithm to 
find patterns in the entries provided. Thus, these algorithms can organize 
samples without having a predefined classification. Non-supervised 
algorithms are useful when it is necessary to discover patterns sets of non- 
classified data. This technique is used in the recognition and identification 
of faces and voices, in addition to the creation of decision-making systems 
in a short time, enabling, for example, the construction of autonomous 
vehicles and drones. In the health area, the technique is used for the 


diagnosis of certain diseases, such as diabetic retinopathy. 
3. VICTOR 


As aforementioned, the Brazilian Justice Systems faces one of the most 
serious “court crisis” of the world. It is not different in STF. In 2018 only, 
there were more than fifty thousand appeals filed before this Court, which 


could decide approximately a hundred and twenty thousand cases a year. 


10 David E. Rumerlhart, Geoffrey E. Hilton, Ronald J. Willinans, Learning 

Representations by back-propagating erros, Nature, v. 323, issue 9, October 1986 

11 Isabela Ferrari, Daniel Becker and Erik Navarro Wolkart, Arbitrium ex machina, Revista 
dos Tribunais, vol. 995/2018, September 2018. 

12 G. G. Gardner et al, Automatic detection of diabetic retinopathy using an artificial 
neural network: a screening tool, British Journal of Ophthalmology, 80, 1996. 


As a comparison, in the 2017-2018 term, the Supreme Court of the United 
States (SCOTUS) agreed to hear 71 cases, and only 69 cases were 


effectively ruled.13 


The first step of the analysis of all the appeals that reach the STF is the 
presence or not of general repercussion, one of its prerequisites. Before 
VICTOR, this analysis was made by civil servants, based on binding 
precedents from the Justices, taking around 40 minutes for each lawsuit. 
Victor does that in 5 seconds. Therefore, VICTOR is apparently a very 


welcomed way out to unburden the STF. 


In relation to its software architecture, VICTOR has several cutting-edge 


technologies embedded and a huge dataset of Court records.‘ 


The dataset used to train VICTOR contains more than a hundred thousand 
lawsuits and almost three million case dockets extracted during the period 
of two years (2017-2019). This data represents approximately four 


terabytes.!° 


Its first challenge was to deal with the fact that the court records reach the 
STF, coming from all the Brazilian Courts (State, Federal, Labour, Military, 
Electoral Justice), in different formats - for example, unstructured PDF 


volume which encloses several documents that have not been indexed. 


Therefore, no matter whether the file was on pdf, jpeg or any other format, 
VICTOR’s first challenge is to be able to read it. To do so, it applies Optical 
Character Recognition (OCR), a technology for recognizing characters from 
an image file or bitmap whether they are scanned, handwritten, typed or 
printed. Thus, through OCR it is possible to obtain a text file machine- 


readable and editable. 
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More than that, VICTOR reorganizes the image it receives, splitting the legal 
reasoning, and highlighting the most important of them, allowing whoever 
has contact with the file later to easily reach the act he or she is searching 


for. 


It is important to understand that, if VICTOR is a key technology that allows 
the system to classify and to structure raw data, it also means that the 
system is analyzing all the lawsuits that reach the Supreme Court, and its 
full content. In a country where most of the cases might reach the Supreme 
Court if the parties want that (and they usually do), this means that VICTOR 


has access to valuable data on Brazil’s legal system. 


How exactly does VICTOR work? The subfield of AI responsible for Victor’s 
architecture is a convolutional neural network (CNN). CNN is a class of 
artificial neural networks of the feed-forward type, which has been 
successfully applied in the processing and analysis of digital images, and in 


experiments involving natural language processing, such as VICTOR.*® 


Python, the programming language, is also a key technology, which allows 
programmers to use different programming styles to create programs, get 
quicker results and write code almost as if speaking in a human language. 
Some of the popular systems and applications that have employed Python 
during development include Google Search, YouTube, BitTorrent et cetera. 
Because Python is a high-level programming language, it abstracts many 
sophisticated details from the programming code. Python focuses so much 
on this abstraction that its code can be understood by most novice 
programmers. Victor also uses Google TensorFlow, which functions by 
sorting through layers of data (also Known as nodes) as part of its learning 
process. In the first layer, the system determines the basic features of the 


object. As deeper movements occur, it looks for more refined information 
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regarding the object. The sorting of images is done at a faster rate, thus 


giving users more valuable information. 


Another technology embedded is XGBoost, which is an optimized distributed 
gradient boosting library designed to be highly efficient, flexible and 
portable. XGBoost provides a parallel tree boosting that solve many data 


science problems in a fast and accurate way. 
4. Beauty? 


Victor undoubtedly is a solution to grant more speed and effectiveness to 
the proceedings instituted before the STF in a more efficient and productive 
model if compared to the traditional evaluation carried out by the court civil 


servants. 


As highlighted in the introduction, VICTOR pilot version aims to distinguish 
the appeals which are apt to be ruled by the STF, regarding one of its 
requirements (the existence of general repercussion, meaning that the topic 
addressed in the appeal has social relevance and, therefore, it should be 
ruled by the STF. 


The categories of appeals that will challenge VICTOR comprise: (i) appeals 
that deal with issues with general repercussion, (ii) appeals that were 
already dismissed due to their lack of general repercussion, (iii) appeals 
regarding which there is not enough information neither to establish the 
presence nor the absence of general repercussion. Regarding the latter, it 
is important to highlight that VICTOR’s database does not comprise subjects 
that have not yet been sufficiently addressed by the Court regarding the 


existence of general repercussion.*/’ 


Somehow, what is being done is nothing more than materializing and 
validating the system of precedents brought by the reform of the Civil 


Procedure Code on 2015. This reform approached the Brazilian Justice 
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System, which has its roots in the civil law tradition, to the common law 
system, such as stare decisis. Speaking about VICTOR, Justice Luiz Fux 
from the STF suggested that by identifying the 27 most recurrent themes 
appealed the High Court can rule approximately ten thousand lawsuits per 


year. 8 


This screening, the main purpose of VICTOR at this initial moment, for the 
purpose of identifying topics of general repercussion was indicated as 84%, 
being able to reach 95%.!9 In addition, while a court clerk takes 44 minutes, 
in general, to evaluate whether there is general repercussion in an appeal, 


Victor spends only a couple of seconds.?° 


It is feasible to conclude that it allows the possibility of the research and 
use of the processed data, facilitating the identification of the parties and 
the object of the appeals, and that it also allows the relocation of court 


clerks in other activities. 
5. Beast? 


According to its developers, VICTOR is a software that does not exercise 
automated decision-making; it only supports court clerks in the triage of 
the appeals. However, experience teaches us that it is not that easy to 
segregate hybrid decisions and the influence of a software support to the 
cognitive activity of human reasoning, as it was shown to the world in the 
United States case Loomis v. Wisconsin, 881 N.W.2d 749 (Wis. 2016), cert. 
denied, 137 S.Ct. 2290 (2017). 


Is it possible to appeal or to file a motion against VICTOR’s general 
repercussion evaluation? Unfortunately, not. There is no provision in 


Brazilian laws for an appeal against this sort of decision. And there is more. 
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Currently, the appellants are not informed when VICTOR is used since its 
pilot-version is randomly picking up appeals to evaluate. This is a violation 
of Brazilian laws, specifically the Brazilian Data Protection Law, which is 
expected to enter into force on May 2021, and provides that automated 


decision-making should be fair, transparent, and informed.?! 


Another serious issue involving VICTOR is related to stakeholders’ hearing 
and its costs. Even though the AI belongs to the STF and, therefore, is 
public, the costs and its code are not. Nobody knows how much it cost for 
the public purse nor how the code works. It lacks the transparency required 
by Brazilian Laws. The first problem is a serious issue of accountability. The 


last one is a technology concern. 


As mentioned in the second chapter, understanding the decision-making 
process and the programming of probabilistic and self-programmable 
algorithms is a Herculean task.” It is said that “developing a machine 
learning algorithm is very difficult; the only thing considered more difficult 


than programming it, is to audit it and explain it” .?3 


AI autonomously modifies its structure while operating, according to the 
data. Therefore, there is a huge gap between the programming activity and 
the behaviour of the AI due to the complexity of its operation; the mere 
observation of the output by a human being - even by its own developer - 
could hardly lead to any conclusion about the internal processes that led 


the inputs to that output, turning the AI a true black box.?4 
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From this modern computational and statistical engineering, serious 
problems may arise involving discrimination, prejudice and bias following 
the opacity and biased datasets of AI systems. Again, it is hard not to 
remember the jeopardy brought by similar software, such as the COMPAS, 


brought to light in the aforesaid Loomis v. Wisconsin case. 
6. Conclusion 
Can we trust Victor? 


According to what has been reported, the degrees of Victor’s precision has 
been very high - higher than if the same activity had been performed by 
humans without the use of these technologies. There is little controversy in 
the idea that the development of technological tools that guarantee 
stability, uniformity, predictability, consistency, and integrity to courts is a 


first-tier urgency for every Justice System in the world, especially Brazil. 


However, Victor can generate incorrect, unjustified, or unfair results 
regarding the deleterious consequences from machine learning from biased 
data sets, to the opacity of these algorithms, and to the discrimination 
potentially generated by them. The autonomy of machine learning 
algorithms turns the tasks they perform difficult to predict and even after 


the decision difficult to explain or rewind the process. 


Right now, the way VICTOR was developed might give space to the 
argument that it is a beast and cannot be trusted. However, it might become 
a beauty if its transparency, accountability, and legibility are improved and 


be object to public and private, both technical and social, scrutiny. 
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